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Abstract 

This paper studies student performance predictions based on the 
United States Medical Licensure Exam (USMLE) Step 1. Subjects 
were second-year medical students from academic years of 2002 
through 2006 (n=711). Three measures of basic science knowledge 
( two curricular and one extracurricular) were evaluated as predictors 
of USMLE Step 1 scores. The USMLE Step 1 scores correlated with 
performance on Organ Systems (r=0.76). Human Structure (r=0.65), 
and CBSE (r=0.69). Accounting for 59% of variance in the USMLE Step 
1 scores. Organ Systems course was a better predictor compared 
to Human Structure (R z =42%), or CBSE (R 2 =51%). Combined, the 
curricular and extracurricular courses accounted for nearly 70% 
of total variance in the Step 1 scores. The study concluded that 
curricular courses are good predictors of student performance on 
the USMLE Step 1, and their value as identifiers of students at risk 
for failure is promising. 


I n 1994, the United States Medical Licensure Examinations (USMLE) 
became the only way in which allopathic physicians could obtain medical 
licensure in the United States (Swanson, Ripkey & Case, 1996). The 
USMLE consists of four separate examinations, Step 1, Step 2 Clinical 
Knowledge (CK), Step 2 Clinical Skills (CS), and Step 3 (Federation of State 
Medical Boards and National Board of Medical Examiners, 2008). Since a 
passing scores on the USMLE Steps 1 and 2 are required for a candidate 
to provide medical care under supervision, i.e., to enter residency training, 
many medical schools have adopted policies that require passage of Steps 1 
and 2 for promotion and graduation. The National Board of Medical Examiners 
(NBME) has not set a limit to the number of times a student attempts to pass 
the USMLE Steps. However, a majority of medical schools and state medical 
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boards in the United States have established a limit to the number of times 
the exams can be taken. 

At many medical institutions, passing the USMLE Step 1 examination is 
required for a student's promotion to the third year clinical clerkships and 
is particularly essential for graduation (Barzansky, Jonas, & Etzel, 1997). 
Our institution has required a passing performance on the USMLE Step 1 
for graduation since 1992, and during the academic year 2001-2002, the 
policy was changed to make that a requirement for promotion to the third 
year clinical clerkships. The attention on student performance on the USMLE 
Step 1 has increased the need to identify accurate predictors of success. 
Several recent studies have revealed a number of variables (learning styles, 
performance on Medical College Admission Test (MCAT), student entry 
grade point average, age, gender, and race) that correlate with USMLE Step 
1 performance (Case, Becker & Swanson, 1993; Elam & Johnson, 1994; 
Koenig, Sireci, & Wiley, 1998; Lynch, Woelfl, Steele & Hanssen, 1998; 
Swanson, Ripkey & Case, 1996; Wiley & Koenig, 1996). With the growing 
significance of students' performance on the USMLE Step 1, research to 
identify significant predictors of success on this standardized test has also 
substantially increased. Several studies have previously identified variables 
such as age, gender, race, learning style, entry level grade point average, the 
performance on Medical College Admissions Test (MCAT), and undergraduate 
and graduate grades that correlate with USMLE Step 1 performance (Case 
et al ., 1993; Elam & Johnson, 1994; Koenig et al ., 1998; Lynch et al., 1998; 
Swanson et al., 1996; Wiley & Koenig, 1996). Use of students' performance in 
the pre-clinical/basic science curricular courses as predictors of USMLE Step 
1 scores has been under-explored although researchers have highlighted its 
importance (Holtman, Swanson, Ripkey, & Case, 2001). Previous research 
has indicated a correlation between students' performance in the medical 
gross anatomy course and the USMLE Step 1 score (Peterson & Tucker, 
2005). In this study we evaluate the ability of two curricular measures and 
one extracurricular measure in predicting performance in the USMLE Step 1 
examination. 

The curricular measures evaluated in this study include the final percent 
scores achieved in two pre-clinical courses, Human Structure in the first 
year curriculum and Organ Systems in the second year curriculum. These 
two curricular measures were selected because they have the highest 
failure rate at our institution in the first two years of the curriculum. The 
Human Structure course is part of the first year curriculum; it integrates 
topics from gross anatomy, microanatomy, and embryology and contains 
gross dissections and microanatomy labs (The University of Toledo College 
of Medicine, 2008a). The Organ Systems course is the principal course in 
the second year of the curriculum. Topics are organized and based on nine 
major organ systems. These include the relevant physiology, pharmacology, 
and pathology for the following systems: Cardiovascular, respiratory, 
renal, electrolytes, hematopoietic, gastrointestinal, hepatic endocrinology, 
reproductive, skin, and skeletal (The University of Toledo College of Medicine, 
2008b). The extracurricular measure evaluated as a predictor of USMLE Step 
1 performance was scores on the Comprehensive Basic Science Examination 
(CBSE). 
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Methodology 

The study was approved by the Institutional Review Board at the 
University of Toledo Health Science Campus (formerly known as the Medical 
College of Ohio). A total of 711 second-year medical students were included 
in this study. A brief description of the study was presented to the students 
at one of the USMLE Step 1 Preparation Program sessions offered by the 
institution's Academic Enrichment Center. The final percent scores for the 
Human Structure and the Organ Systems courses were obtained from the 
respective course directors. All students were required to take the paper and 
pencil version of the CBSE and were allowed a maximum of three hours to 
complete the test. The USMLE Step 1 was administered at one of the NBME 
approved testing sites of the student's choice. The majority of students opted 
to take their USMLE Step 1 at the institution's Academic Test Center, which 
is one of eight medical school testing sites in the nation that is approved by 
the NBME. Only the first time USMLE Step 1 scores obtained by the students 
were used in the analysis. 

Data from five consecutive academic years (2002 through 2006) was 
combined for analysis (n = 711). All the identifiers in the data were removed 
in order to maintain confidentiality. Statistical analysis included bivariate 
Pearson's correlation to test the strength of associations between the 
measures. Linear regression was performed using each of the three different 
measures separately and in combinations to identify the model that best 
predicts students' performance on the USMLE Step 1. Multivariate normality 
(normal distribution of the dependent variable for each combination of 
values of the predictors) was examined prior to the regression analysis. 
Multi-collinearity of the predictors was addressed using the standardized 
(z) scores in the analyses. The predictability (i.e., the ability to explain the 
variance in the dependent variable USMLE Step 1 score) was interpreted 
based on the value of adjusted coefficient of determination (R2). For all 
the models, quadratic and interaction terms were examined to increase 
the predictability. Significance of the difference between R2 values from 
different models were examined using the F statistic. Statistical analyses 
were performed using SAS 9.1 (SAS Institute, Cary, NC, U.S.A.). 

Results 

Descriptive statistics of the combined data from a total of 711 second- 
year medical students from five academic years (2002 through 2006) are 
presented in Table 1. The mean USMLE Step 1 score was 214.2 ± 21, the 
CBSE was 63.0 ± 7, and the final average percent scores in the Organ 
Systems and the Human Structure courses were 82.7 ± 7 and 80.9 ± 7 
respectively. There were no statistically significant differences in the mean 
scores of the three measures across all five academic years. 
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Table 1. 

Description of the curricular and extra-curricular measure scores from 
academic years 2002 to 2006. 


2002 - 2006 
(n=711) 

Mean ± SD 

Organ Systems Final Percent 

82.89 ± 7.0 

Human Structure Final Percent 

81 .06 ±7.1 

CBSE score 

63.44 ± 7.0 

USMLE Stepl score 

214.6 ±21.0 


SD = standard deviation 


All three measures significantly correlated with each other and with 
the Step 1 scores. A strong positive correlation (r=0.74, p<0.001) was 
observed between the two curricular measures. The strongest correlation 
was observed between the Step 1 scores and the performance in the Organ 
Systems course (r=0.76, p<0.001). Performance in the Human Structure 
course and the CBSE scores correlated moderately with the Step 1 scores 
(r=0.65, p<0.001 and r=0.69, p<0.001 respectively). 

Table 2 illustrates the results from the simple regression analysis. 
Performance in each of the two pre-clinical curricular courses significantly 
predicted students' performance on the USMLE Step 1 examination. Students' 
final percent score in the Organ Systems course accounted for 59% of the 
variance in the Step 1 score and was a better predictor than the score in 
Human Structure course (42%). The CBSE course accounted for 51% of the 
variability in the Stepl scores. 

Table 2. 

Simple regression predicting the USMLE Step 1 score* 



P 

95% Cl 

p-value 

R 2 (%), 

Intercept 

Organ Systems 

2.44 

2.29-2.59 

<0.001 

59.6,11.21 

Human Structure 

1.91 

1.74-2.08 

<0.001 

42.3, 59.64 

CBSE 

2.16 

1.99-2.32 

<0.001 

51.1,77.42 


’All regression models were significant at the 0.01 level (2 sided) 

P = Regression coefficient; Cl = confidence interval; R 2 = Coefficient of 
determination 
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Table 3 presents the multiple regression analysis with the two curricular 
courses in one model and together with the extra-curricular measure in 
a separate model. Predictability combining the final percent scores in the 
Human Structure and Organ Systems courses (adjusted R2=0.61) was 
noted as marginally better than that from the Organ Systems alone (R2 
change=0.018, F change=30.17, pcO.OOl). However, using the score on 
the CBSE as an additional predictor along with both the curricular courses, 
a total of 69% of variance in the USMLE scores could be explained (R2 
change=0.085, F change=180.93, p<0.001). All three measures significantly 
contributed to the overall predictability. No difference in the R2 value was 
noted in separate regression models that used the standardized scores or 
with the academic year as an additional variable. Use of the quadratic and 
interaction terms did not significantly contribute to the final models. 

Table 3. 


Multiple regression predicting the USMLE Step 1 score* 


MODEL 

P 

95% Cl 

P- 

value 

Adj. R 2 (%), 
Intercept 

Power 

Standardized 

estimate 

Organ Systems 

1.97 

1.76-2.19 

<0.001 

61.2,4.14 

0.99 

0.631 

Human Structure 

0.57 

0.37-0.77 

<0.001 


0.99 

0.193 

Organ Systems 

1.48 

1.27-1.69 

<0.001 


0.99 

0.474 

Human Structure 

0.31 

0.13-0.49 

0.001 

69.7, -4.29 

0.91 

0.107 

CBSE 

1.11 

0.95-1.27 

<0.001 


0.99 

0.369 


’All regression models were significant at the 0.01 level (2 sided) 

P = Regression coefficient; Cl = confidence interval; R 2 = Coefficient of determination 


With information from the regression analysis comprising all three 
predictors, an equation to obtain predicted USMLE Step 1 score was derived 
as follows: Predicted USMLE Stepl score = 1.48 (Organ Systems %) + 0.31 
(Human Structure %) + 1.11 (CBSE raw score) - 4.29. Using this equation 
nearly 70% of the variability in the USMLE Step 1 score could be predicted. 

Discussion 

In this study, performance of students in two curricular courses and the 
comprehensive basic science examination was used to predict the USMLE 
Step 1 scores. To the best of our knowledge this is the first study to utilize 
a system-based curricular course such as Organ Systems to predict Step 
1 performance. Our hypothesis was that the final percent achieved in 
the two curricular courses that have the highest failure rates in the pre- 
clinical curriculum at our institution would be good predictors of Step 1 
performance. In addition, we selected the CBSE, a commercially available 
test that evaluates basic science knowledge, as an extracurricular predictor 
of Step 1 performance. 
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Results of this study demonstrated that all the evaluated measures were 
statistically significant predictors of USMLE Step 1 performance. Of the two 
pre-clinical courses in the College of Medicine curriculum, the final percent 
score in the Organ Systems course was a better predictor of USMLE Step 
1 performance than that in the Human Structure course. At our institution, 
the Human Structure course is included in the first year curriculum while the 
Organ Systems course is part of the second year curriculum. The difference 
between the two curricular courses as predictors could be attributed partly 
to the distinct order in which the courses are offered in the pre-clinical years 
of medical education. Our findings also verify that the USMLE Step 1 practice 
test, CBSE, was a good independent predictor of Step 1 performance. The 
variability in the Step 1 scores explained by the CBSE in our study was 
similar to that reported in other studies (Elam & Johnson, 1994). 

As single predictors, the final percent scores in the Organ Systems 
explained the maximum variability in Step 1 performance followed by the 
CBSE scores and the final percent scores in the Human Structure course. 
It can be suggested that along with the Step 1 practice tests, student 
achievement in the curriculum could be used to predict performance on 
the USMLE Step 1 examination. Combining the students' performance on 
the CBSE and in their two curricular courses increased the ability to predict 
USMLE performance and explained approximately 70% of total variance in 
the USMLE scores. This result suggests that institutional courses are good 
predictors of student performance on national standardized tests, and it 
is plausible that the teaching methodologies, format of the course, and/or 
learning methods in these courses play an indirect but significant role in the 
prediction. 

Recognition of curricular courses as predictors of USMLE Stepl 
performance has several implications. It provides an opportunity for the 
educational authorities and committees at medical institutions to direct 
existing resources or dedicate additional efforts to improve students' 
performance in these courses, thereby ensuring better performance on the 
USMLE Step 1. With institution-specific prediction models using curricular 
courses, early identification of students at risk for failing the Step 1 could 
be possible. This provides an opportunity to intervene appropriately via 
intensive study strategies, tutoring and/or review programs that promise 
better USMLE Step 1 performance from the students. 

Past research has suggested several different predictors of USMLE Step 1 
performance including age, gender, race, learning style of the students, and 
preadmission variables such as entry level grade point average, undergraduate 
grades, MCAT performance, etc (Basco, Way, Gilbert & Hudson, 2002; 
Case, Becker & Swanson, 1993; Elam & Johnson, 1994; Kleshinski, Khuder, 
Shapiro & Gold, 2007; Koenig, Sireci & Wiley, 1998; Lynch, Woelfl, Steele & 
Hanssen, 1998; Wiley & Koenig, 1996). We acknowledge that our study lacks 
information on these predictors; however, earlier studies have indicated that 
many of these variables predict students' performance in the pre-clinical 
years of medical education (Haist, Wilson, Elam, Blue & Fosson, 2000; Hoschl 
& Kozeny, 1997; Julian, 2005). Therefore, it is plausible to assume that the 
observed predictability of basic science curricular courses in this study is 
influenced by other predictors as indicated above. Nevertheless, a student's 
performance in the curricular courses could serve as a convenient, reliable, 
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and convincing predictor of the USMLE Step 1 score. Other limitations of this 
study are that the data included in the analysis was from a single medical 
institution, and the sequence in which the two curricular courses are offered 
to the students could have influenced the predictions. As specified earlier, 
the basis for selecting the Organ Systems and Human Structure courses in 
our analysis was the high failure rate noted among the students over the 
past five years. A lack of variability in students' performance in the other 
courses of the first and second year curriculum accounted for their exclusion 
in this study. One of the strengths of this study includes a large sample 
of medical students included from multiple consecutive academic years. In 
addition, performance on the USMLE Step 1 was assessed by the actual 
score on the test with linear data analysis, not by pass or fail based on a 
cutoff score. 


Implications and Further Study 

Implications of this study can be directed towards the role of learning 
centers in academic institutions, their influence on student performance in 
curricular courses, and ultimately on the national licensure examinations. 
Future research efforts should focus on developing strategies that improve 
student curricular performance and methods to evaluate their outcome on 
the standardized examination. 


Conclusion 

Our study supports that select pre-clinical curricular courses of medical 
education are convenient and good predictors of student performance on 
the USMLE Step 1. Efforts for improving student performance in these 
courses will culminate in better performance on the standardized licensing 
examination. Some of the techniques that could potentially improve student 
curricular performance are the following: types of tutoring (supplemental 
instruction, structured learning assistance programs, individual and group 
tutoring) and a variety of teaching methods that take into account multiple 
types of learning styles. Additionally, this would also increase students' 
self-esteem, self-confidence, and future accomplishment throughout their 
medical careers. Student success on the USMLE Step 1 improves the overall 
institutional performance which is important for accreditation of medical 
schools by the Liaison Committee for Medical Education, as well as for the 
ability of the institution to attract highly qualified students and residents. 
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